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ABSTRACT 

Objectives: To investigate the nature and potential 
implications of under-reporting of morbidity 
information in administrative hospital data. 
Setting and participants: Retrospective analysis of 
linked self-report and administrative hospital data for 
32 832 participants in the large-scale cohort study 
(45 and Up Study), who joined the study from 2006 to 
2009 and who were admitted to 313 hospitals in New 
South Wales, Australia, for at least an overnight stay, 
up to a year prior to study entry. 
Outcome measures: Agreement between self-report 
and recording of six morbidities in administrative hospital 
data, and between-hospital variation and predictors of 
positive agreement between the two data sources. 
Results: Agreement between data sources was good for 
diabetes (k=0.79); moderate for smoking (k=0.59); fair 
for heart disease, stroke and hypertension (k=0.40, 
k=0.30 and k =0.24, respectively); and poor for obesity 
(k=0.09), indicating that a large number of individuals 
with self-reported morbidities did not have a 
corresponding diagnosis coded in their hospital records. 
Significant between-hospital variation was found (ranging 
from 8% of unexplained variation for diabetes to 22% for 
heart disease), with higher agreement in public and large 
hospitals, and hospitals with greater depth of coding. 
Conclusions: The recording of six common health 
conditions in administrative hospital data is highly 
variable, and for some conditions, very poor. To support 
more valid performance comparisons, it is important to 
stratify or control for factors that predict the 
completeness of recording, including hospital depth of 
coding and hospital type (public/private), and to increase 
efforts to standardise recording across hospitals. Studies 
using these conditions for risk adjustment should also be 
cautious of their use in smaller hospitals. 
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INTRODUCTION 

Most nations with advanced economies pub- 
licly report on the comparative performance 
of hospitals with a view to accelerating and 
informing efforts to improve quality and allow- 
ing patients to make informed choices. 



Strengths and limitations of this study 



The study was based on linked data from a 
large-scale cohort study and administrative hos- 
pital data to evaluate four health conditions and 
two health risk factors, as well as their 
combinations. 

The study used advanced multilevel modelling 
methods to comprehensively evaluate the record- 
ing of each morbidity in administrative data and 
quantify between-hospital variation. 
The study provides detailed information about 
how the validity of morbidity reporting varies 
among hospitals after accounting for patient 
factors. 

Limitations include the use of self-reported data 
in the absence of a 'gold standard' such as 
medical records. 



Diagnoses recorded in administrative hospital 
data are commonly used in the construction 
and case-mix adjustment of hospital perform- 
ance metrics, as well as for risk adjustment in 
epidemiological studies. 

The construction of reliable health metrics 
relies on statistical methods that take into 
account the degree to which patients treated 
in different facilities have different morbidity 
and risk profiles that predispose them to 
requiring different interventions or to achiev- 
ing different outcomes. These statistical 
methods, known as case-mix or risk adjust- 
ment, account for patient-related factors that 
are above and beyond the immediate control 
of healthcare professionals. 

Thus, properly constructed performance 
metrics fairly reflect differences in healthcare 
experiences, patient outcomes and risks of 
adverse events. There has been some criti- 
cism of case-mix adjustments because they 
are subject to measurement error, 1 but 
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case-mix adjustment is still considered to be less biased 
than unadjusted comparisons. 2 

Most methods of case-mix adjustment rely principally 
on demographic and diagnostic information that is cap- 
tured in administrative hospital data collections. The 
hospital data are collected and recorded in a database 
for administrative purposes, with clinical coders coding 
diagnostic information based on the patient's medical 
records. 3 This approach may be suboptimal 4 5 because 
evidence from many countries suggests that administra- 
tive hospital data under-report the morbidity informa- 
tion needed to fully account for differences between 
hospitals in patient-related factors that predispose them 
to differences in measured outcomes. 6-13 However, the 
impact of this under-reporting on comparative measures 
of hospital performance depends on whether it varies 
systematically among hospitals, because of differences in 
factors such as training or practice among coding staff, 
the comprehensiveness of clinicians' notes or 'upcoding' 
relating to funding models or incentives. 14 

This issue is relatively unexplored, aside from the work 
by Mohammed et al, 2 which reported a non-constant rela- 
tionship between case-mix variables and mortality among 
hospitals in the UK, explained by differences in clinical 
coding and admission practices across hospitals. These var- 
iations in coding accuracy were shown to be related to geo- 
graphic location and bed size, with small rural facilities 
performing better than large urban hospitals. 15 16 In 
Australia, variations in the reporting and coding of second- 
ary diagnoses in administrative hospital data have been 
shown to exist in public hospitals among Australian 
states, 17 and also among hospitals within the state of New 
South Wales (NSW), with greater under-reporting in 
private and rural hospitals. 3 However, the relative contribu- 
tions of patient and hospital factors to these variations 
have not been identified, nor has this variation been for- 
mally quantified. 

This study, using data linkage of survey and administra- 
tive data, aimed to further investigate the nature and 
potential implications of under-reporting of morbidity 
information in administrative hospital data by: (1) measur- 
ing the agreement between self-reported morbidity infor- 
mation and coded diagnoses; (2) quantifying the amount 
of between-hospital variation in this agreement and (3) 
identifying patient and hospital characteristics that predict 
higher or lower levels of agreement. We focused on clin- 
ical conditions common to case-mix and risk-adjustment 
models — diabetes, heart disease, hypertension and stroke. 
We also focus on smoking and obesity, due to their impact 
on health trajectories, rapid shifts in prevalence, substan- 
tial geographic variation in rates 18 and paucity of inter- 
national evidence on completeness of coding. 

METHODS 
Data sources 

The 45 and Up Study 

The 45 and Up Study is a large-scale cohort study involving 
267 153 men and women aged 45 years and over from the 



general population of NSW, Australia. The study is 
described in detail elsewhere. 19 Briefly, participants in the 
45 and Up Study were randomly sampled from the data- 
base of Australia's universal health insurance provider, 
Medicare Australia, which provides near-complete coverage 
of the population. People 80+ years of age and residents of 
rural and remote areas were oversampled. Participants 
joined the Study by completing a baseline questionnaire 
(available at https: / / www.saxinstitute.org.au/ our-work/ 
45-up-study/ questionnaires/) between January 2006 and 
December 2009 and giving signed consent for follow-up 
and linkage of their information to routine health data- 
bases. About 18% of those invited participated and partici- 
pants included about 11% of the NSW population aged 
45 years and over. 19 

The NSW Admitted Patient Data Collection 

The Admitted Patient Data Collection (APDC) includes 
records of all public and private hospital admissions 
ending in a separation, i.e. discharge, transfer, type- 
change or death. Each separation is referred to as an 
episode of care. Diagnoses are coded according to the 
Australian modification of the International Statistical 
Classification of Diseases and Health Related Problems 
10th Revision (ICD-10-AM). 20 Up to 55 dia enoses codes 
are recorded on the APDC, including the principal diag- 
nosis and up to 54 additional diagnoses. Additional diag- 
noses are defined as "a condition or complaint either 
coexisting with the principal diagnosis or arising during 
the episode of care" in the Australian Coding Standards 
and should be interpreted as conditions that affect 
patient management. 21 Assignment of diagnosis codes is 
done by trained clinical coders, using information from 
the patient's medical records. 

The APDC from 1 July 2000 to 31 December 2010 was 
linked probabilistically to survey information from the 
45 and Up Study by the NSW Centre for Health Record 
Linkage (http://www.cherel.org.au) using the 'best prac- 
tice' protocol for preserving privacy. 22 

Study population 

The study population comprised patients aged 45 years 
and above who participated in the 45 and Up Study and 
who had a hospitalisation lasting at least one night in 
the period up to 365 days prior to filling out the base- 
line 45 and Up Study survey. Day stay patients were 
excluded from the analysis to make the study more 
robust and generalisable beyond NSW and Australia, as 
there are differences in admission practices for same day 
patients between Australia and most other comparable 
countries. 23 NSW is home to 7.4 million people, or 
one-third of the population of Australia. 

Measuring morbidity 

We examined four health conditions (diabetes, heart 
disease, hypertension and stroke) and two health risk 
factors (obesity and smoking), referred to hereafter col- 
lectively as 'morbidities'. For each participant, these 
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health conditions were measured using self-report and 
administrative hospital data. 

Self-reported morbidities were ascertained on the 
basis of responses to questions in the baseline 45 and 
Up Study survey. Diabetes, hypertension, stroke and 
heart disease were identified using the question "Has a 
doctor ever told you that you have [name of condi- 
tion]?" Participants who did not answer the question 
were excluded from analyses (n=1242). 

Smoking was classified on the basis of answering "yes" 
to both of the questions "Have you ever been a regular 
smoker?" and "Are you a regular smoker now?" 
Participants' responses to the questions "How tall are you 
without shoes?" and "About how much do you weigh?" 
were used to derive body mass index (BMI), defined as 
body weight divided by height squared (kg/m 2 ). The 
WHO's 24 classification system was used to categorise indi- 
viduals as obese (BMI >30 kg/m 2 ). 

Morbidity information in administrative hospital data 
was ascertained using all 55 diagnosis codes in the 
APDC records (ICD-10-AM: E10-E14 for diabetes, 120- 
152 for heart disease, 160-169, G45, G46 for stroke, 110- 
115 and R03.0 for hypertension, F17.2 or Z72.0 for 
smoking and E66 for obesity) . The inclusion of broader 
ICD-10-AM codes for heart disease and stroke was 
chosen because of the broad definition of disease type 
in the self-reported data. Thus, heart disease codes were 
inclusive of coronary heart disease, pulmonary heart 
disease and other forms of heart diseases, including 
heart failure and arrhythmias. Stroke codes included 
cerebrovascular diseases without infarction among 
others. 

Predictors of agreement 

We explored patient-level as well as hospital-level factors 
as predictors of agreement between the two data 
sources. 

Patient-level factors were self-reported in the 45 and 
Up Study baseline survey and included age, sex, educa- 
tion, country of birth, income and functional limitation. 
Functional limitation was measured using the Medical 
Outcomes Study-Physical Functioning scale, 25 and classi- 
fied into 5 groups: no limitation (score of 100), minor 
limitation (score 95-99), mild limitation (score 85-94), 
moderate limitation (60-84) and severe limitation (score 
0-59). 

Facility-level factors were type of hospital (public/ 
private), hospital peer group (akin to hospital size 
defined by number of case-mix weighted separations, 26 
which includes hospital remoteness in the classification), 
remoteness of hospital and depth of coding. 
Remoteness of the Statistical Local Area in which the 
hospital was located was classified according to the 
Accessibility/Remoteness Index of Australia (ARIA+), 
grouped into four categories (major city, inner regional, 
outer regional, remote/very remote). 27 Depth of hos- 
pital coding was the mean number of additional diagno- 
ses coded per episode of care for each hospital, 



calculated using all overnight hospitalisations for the full 
45 and Up Study cohort from 2000 to 2010, and divided 
into four groups at the 25th, 50th and 75th centile. 
Hospital peer groups were divided into 5 categories: 
principal referral (>25 000 separations per year), major 
(10 000-24 999 separations per year), district (2000- 
9999 separations per year), community (up to 2000 
separations per year) and other (non-acute, unpeered 
hospitals) . Missing information was treated as a separate 
category for any variables with missing data. 

Statistical methods 

We examined patient-level agreement between data 
sources for each of the six morbidities individually, as 
well as for their 15 two-way combinations. We compared 
the self-reported responses (yes/no) with all the diagno- 
ses provided in the hospital records both for 'index' 
admissions and for the 'lookback' period admissions. 28 
The 'index' admission was the overnight hospital stay 
with admission date closest to the survey completion 
date and no longer than a year prior. Morbidity was 
coded as 'yes' if any of the diagnoses during that stay 
contained a mention of that morbidity. The 'lookback' 
admissions included all overnight stays in the 365-day 
period that preceded and included the 'index' admis- 
sion. Morbidity was coded as 'yes' if any of the diagnoses 
from any lookback admissions contained a mention of 
that morbidity. 

Agreement between the two data sources (yes/no) was 
measured using Cohen's k statistic, k Values above 0.75 
denote excellent agreement, 0.40-0.75 fair to good 
agreement and below 0.45 poor agreement. 29 Agreement 
was computed for all 313 hospitals in the state, regardless 
of size, as well as for the 82 largest public hospitals, for 
which performance metrics are publicly reported. 

Multilevel logistic regression was used to estimate OR 
with 95% CIs for patient-level and hospital-level factors 
that predicted positive agreement between the two data 
sources. Multilevel models were chosen because of the 
clustering of patients within hospitals. Models were run 
for each of the six morbidities separately. These analyses 
were constrained to only those participants who self- 
reported the morbidity of interest, and the outcome was 
whether the index hospital record contained a mention 
of the morbidity or not. Addition of the hospital-level 
characteristics was done one at a time, due to the collin- 
earity between variables. All ORs presented are adjusted 
for all other demographic variables in the model. 

Variation at the hospital level was expressed as a 
median OR (MOR), which is the median of the ORs of 
pairwise comparisons of patients taken fr om ra ndomly 
chosen hospitals, calculated as e xp°- 954x ^ variance ; 30 and 
the intraclass correlation coefficient (ICC), which is the 
percentage of the total variance attributable to the hos- 
pital level. 31 Large ICCs indicate that differences among 
hospitals account for a considerable part of the variation 
in the outcome, whereas a small ICC means that the 
hospital effect on the overall variation is minimal. The 
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relative influence of the hospital on reporting of mor- 
bidity was calculated using a variance partitioning coeffi- 
cient expressed as a percentage of the total variance 
using the Snijders and Bosker latent variable 
approach. 31 

All data management was done using SAS V.9.2 32 and 
multilevel modelling using MLwiN V.2.24. 33 

The conduct of the 45 and Up Study was approved by 
the University of New South Wales Human Research 
Ethics Committee (HREC). 



RESULTS 

Descriptive characteristics 

A total of 32 832 study participants were admitted to 313 
hospitals up to a year prior to completing the 45 and 
Up Study baseline survey. Just over half of the index 
admissions (53%) were planned stays in hospital, and 
57% were to a public hospital. Around one-third of the 
index admissions occurred within the 3 months before 
study entry, and the mean length of stay was 4.8 days 
(median=3 days). Just under half of the sample (47%) 
reported having hypertension, with heart disease or 
obesity reported by 25% and current smoking by 6.1% 
of the sample. One-third (34%) of participants had two 
or more morbidities (data not shown). Other character- 
istics of the sample at their index admission are shown 
in table 1. Characteristics of hospitals are summarised in 
table 2. 



Concordance between self-report and hospital records 

Overall, reporting of morbidity differed between the two 
data sources with 23 257 (71%) participants having at 
least one of the six self-reported morbidities, and 1 1 977 
(36.5%) and 14 335 (43.7%) of the sample having at 
least one morbidity recorded on their index or lookback 
hospital admissions, respectively. 

Table 3 gives the summary concordance measures for 
each morbidity and two-way morbidity combination. For 
the index admission, good agreement was found for dia- 
betes (k=0.79); moderate agreement for smoking 
(k=0.59); fair agreement for heart disease (k=0.4); and 
poor agreement for stroke (k=0.3), hypertension (k=0.24) 
and obesity (k=0.09). In two-way combinations, moderate 
levels of agreement were found only for diabetes combina- 
tions (with smoking, hypertension and heart disease). 

Incorporating a 1-year lookback period increased the 
numbers of participants with a morbidity recorded in a 
hospital record, with average relative increases in the k 
values of 20% (ranging from 2% increase for smoking 
to 41% increase for obesity). Good to excellent level of 
agreements were still found only for diabetes (k=0.83) 
and smoking (k=0.6). 

Agreement was only slightly higher among the 82 
large public hospitals (see online supplementary table 
SI) with relative k values higher by 4%, on average. 



Table 1 Characteristics of the study sample at their index 




admission 



All participants 
(N=32 832) 

N Per cent 


Demographic characteristics 






Sex 






Male 


16812 


51.2 


Female 


16 020 


48.8 


Age 






45-59 


9666 


29.4 


60-79 


16 624 


50.6 


80+ 


6540 


19.9 


Country of birth 






Australia 


25 001 


76.2 


Other 


7448 


22.7 


Unknown 


383 


1.2 


Highest education level 






No school 


5196 


15.8 


Year 10 or equivalent 


7894 


24.0 


Year 12 or equivalent 


2975 


9.1 


Trade 


4270 


13.0 


Certificate 


6109 


18.6 


University degree 


5662 


17.3 


Unknown 


726 


2.2 


Household income ($, per annum) 






<20 000 


9077 


27.7 


20 000-<50 000 


8223 


25.1 


500 000-<70 000 


2560 


7.8 


70 000+ 


5042 


15.4 


Not disclosed 


6003 


18.3 


Missing 


1927 


5.9 


Functional status 






No limitation 


4915 


15.0 


Mild limitation 


6011 


18.3 


Moderate limitation 


8701 


26.5 


Severe limitation 


10 121 


30.8 


Missing 


3084 


9.4 


Admission characteristics 






Admission type 






Surgical 


15 464 


47.1 


Other 


1439 


4.4 


Medical 


15 929 


48.5 


Emergency status 






Emergency 


13 484 


41.1 


Planned 


17 544 


53.4 


Other 


1803 


5.5 


Hospital of admission 






Hospital type 






Public 


18 734 


57.1 


Private 


14 096 


42.9 


Hospital remoteness 






Major city 


19 754 


60.2 


Inner regional 


8424 


25.7 


Outer regional 


4137 


12.6 


Remote/very remote 


363 


1.1 


Hospital depth of coding 






1 — least comprehensive 


1629 


5.0 


2 


8803 


26.8 


3 


11 543 


35.2 


4 — most comprehensive 


10 857 


33.1 


Hospital peer group 






Principal referral 


6329 


19.3 


Major 


11 052 


33.7 


District 


6862 


20.8 


Community 


7018 


21.4 


Other 


1571 


4.8 
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Table 2 Characteristics of the hospital of admission 

All hospitals (N=313) 
N Per cent 



Hospital type 



Pi ihlin 


00A 


71 R 
/ I .D 


r 1 1 vdic 


oo 


9ft 1 


Hncnitpl rpmntpnPCQ 
nuo|jiicii idiiiuidiicoo 






l\/laior pit\/ 
ivictjui uiiy 


1 9 A 




ii ii ici i tJyiui idi 


79 


9^ n 




QA 


OU.U 


Rpmntp/\/pr\/ rpmntp 
pici i iuic/ vci y iciiiuic 


90 




Hospital depth of coding 






1 — least comprehensive 


48 


15.3 


2 


91 


29.1 


3 


89 


28.4 


4 — most comprehensive 


85 


27.2 


Hospital peer group 






Principal referral 


14 


4.5 


Major 


33 


10.5 


District 


51 


16.3 


Community 


121 


38.7 


Other 


94 


30.0 



Patient-level and hospital-level predictors of positive 
agreement 

The patient factors, which predicted positive agreement 
between the two data sources, differed between morbid- 
ities (table 4). Male sex was associated with better agree- 
ment for diabetes (OR=1.37, 95% CI 1.19 to 1.58), heart 
disease (OR=1.30, 95% CI 1.17 to 1.44) and hyperten- 
sion (OR=1.28, 95% CI 1.18 to 1.38; see online supple- 
mentary table S2). 

Older patients were significantly less likely to have 
smoking (80+years OR=0.48, 95% CI 0.31 to 0.74) and 
obesity (OR=0.14, 95% CI 0.08 to 0.26) recorded in their 
hospital records and significantly more likely to have 
hypertension recorded (OR=1.32, 95% CI 1.16 to 1.49), 
compared with younger patients (45-59 years). People 
with higher levels of functional limitation were signifi- 
cantly more likely to have hypertension, diabetes and 
obesity recorded on their most recent hospital stay. 
Planned admissions to hospital had lower odds of having 
any of the six conditions recorded, as did medical admis- 
sions (for diabetes, smoking and obesity only). Agreement 
did not vary significantly for any other patient factors. 

The four hospital-level covariates (hospital type, hos- 
pital peer group, hospital remoteness and depth of 
coding) were added to multilevel models (including a 
random intercept for hospital) one at a time, separately. 
Positive agreement between self-report and hospital 
records was significantly lower for hospitals with lower 
depth of coding across all morbidities. The odds of 
recording were also lower among private hospitals for all 
six morbidities, with this difference being statistically sig- 
nificant for hypertension, heart disease and stroke only. 
Records from smaller hospitals (district and community 
peer groups) were significantly less likely to agree with 



self-reported data on hypertension, diabetes and heart 
disease. Positive agreement did not vary significantly 
with remoteness of hospital, with the exceptions of dia- 
betes (lower agreement for outer regional, remote and 
very remote hospitals) and smoking (lower agreement 
for remote and very remote hospitals; see online supple- 
mentary table S3) . 

Quantifying variation between hospitals 

Before any hospital-level variables were added into the 
multilevel model, ICC indicated that between 8% (dia- 
betes) and 22% (heart disease) of the residual (unex- 
plained) variation in agreement was attributable to the 
hospital after adjustment for the patient-level factors 
(table 5). This equated to MORs of 1.64 and 2.48, 
respectively, indicating that a patient in one hospital had 
an average of between 64% and 148% higher odds of 
having a particular morbidity recorded than a patient in 
a hospital with lower levels of recording. Less variation at 
the hospital level was found for the recording of diabetes, 
smoking and stroke, while more variation at the hospital 
level was found for the recording of hypertension, heart 
disease and obesity. When the analyses were restricted to 
82 large public hospitals only, the between-hospital vari- 
ation decreased to between 2% (stroke) and 13% (hyper- 
tension), or MOR of 1.24 and 1.94 (figure 1). This 
between-hospital variation was still significant for all mor- 
bidities except for stroke. Between-hospital variation was 
further reduced once lookback admissions were used to 
identify morbidities. 

The addition of hospital-level variables to multilevel 
models, one at a time, separately, helped ascertain which 
factors explained the variation between hospitals (table 5). 
The addition of hospital-level factors contributed to 
explaining (i.e. decreasing) the residual variation for all 
conditions, except obesity. For the other morbidities, dif- 
ferences in the depth of coding explained from 16% 
(smoking) to 42% (hypertension) of residual variation 
between hospitals, while hospital type (public/private) 
explained from 0% (smoking) to 59% (stroke), and hos- 
pital peer group explained from 10% (hypertension) to 
27% (diabetes) residual variation between hospitals. 

DISCUSSION 

Our study found that the concordance of administrative 
hospital and self-reported data varied between the six 
morbidities examined, with agreement ranging from 
good for diabetes; moderate for smoking; through to 
fair for heart disease; and poor for hypertension, stroke 
and obesity. We demonstrated considerable between- 
hospital variation in the recording of these common 
health conditions. Smaller, but still significant, between- 
hospital variation was found when restricting the ana- 
lyses to the largest public hospitals in the state. 

Previous studies have validated information recorded 
in NSW administrative hospital data for demographic 
factors, 34 35 and recording of perinatal conditions, 36-39 
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Table 3 Agreement measures between self-report and hospital data, index and lookback admissions, all public and private hospitals in New South Wales, Australia 
(n=313) 






Index admission 










Lookback admissions 










45 and Up yes 


45 and Up no 






45 and Up yes 


45 and Up no 








APDC 


APDC 


APDC 


APDC 


Per 




APDC 


APDC 


APDC 


APDC 


Per 




Morbidities* 


yes 


no 


yes 


no 


cent 


95% CI 


yes 


no 


yes 


no 


cent 


95% CI 


Hypertension 


4767 


10512 


1434 


16 119 


24.0 


(22.9 to 25.0) 


6260 


9019 


2051 


15 502 


30.2 


(29.1 to 31 .2) 


Heart disease 


3639 


4668 


1942 


22 583 


40.3 


(39.0 to 41 .5) 


4673 


3634 


2697 


21 828 


47.0 


(45.8 to 48.2) 


Diabetes 


3560 


1234 


347 


27 691 


79.1 


(78.1 to 80.1) 


3928 


866 


479 


27 559 


83.0 


(82.1 to 83.9) 


Stroke 


541 


1939 


306 


30 046 


29.8 


(27.0 to 32.6) 


776 


1704 


488 


29 864 


38.3 


(35.8 to 40.8) 


Smoking 


1205 


804 


727 


30 096 


58.7 


(56.7 to 60.7) 


1411 


598 


1076 


29 747 


60.1 


(58.2 to 61 .9) 


Obesity 


551 


7611 


114 


24 556 


9.1 


(7.3 to 10.9) 


810 


7352 


209 


24 461 


12.8 


(11.1 to 14.6) 


Hypertension+heart 


1172 


3481 


1270 


26 909 


25.8 


(23.8 to 27.7) 


1807 


2846 


2008 


26171 


34.3 


(32.6 to 36.0) 


disease 


























Hypertension+diabetes 


1819 


1238 


759 


29 016 


61.3 


(59.6 to 62.9) 


2186 


871 


1021 


28 754 


66.6 


(65.2 to 68.1) 


Hypertension+stroke 


203 


1317 


189 


31 123 


19.7 


(15.7 to 23.7) 


329 


1191 


340 


30 972 


28.0 


(24.5 to 31 .5) 


Hypertension+smoking 


133 


598 


180 


31 921 


24.5 


(19.2 to 29.7) 


199 


532 


319 


31 782 


30.6 


(26.0 to 35.2) 


Hypertension+obesity 


234 


4574 


93 


27 931 


7.4 


(4.9 to 9.8) 


383 


4425 


183 


27 841 


11.5 


(9.2 to 13.9) 


Heart disease+diabetes 


646 


1154 


404 


30 628 


43.0 


(40.3 to 45.8) 


904 


896 


661 


30 371 


51.2 


(48.9 to 53.6) 


Heart disease+stroke 


76 


973 


126 


31 657 


11.2 


(6.1 to 16.4) 


149 


900 


261 


31 522 


19.0 


(14.4 to 23.5) 


Heart disease+smoking 


76 


294 


222 


32 240 


22.0 


(15.3 to 28.6) 


118 


252 


373 


32 089 


26.5 


(20.8 to 32.2) 


Heart disease+obesity 


79 


1938 


79 


30 736 


6.4 


(2.5 to 10.4) 


151 


1866 


169 


30 646 


11.4 


(7.7 to 15.2) 


Diabetes+stroke 


85 


555 


58 


32 134 


21.1 


(15.0 to 27.3) 


140 


500 


119 


32 073 


30.4 


(24.9 to 35.8) 


Diabetes+smoking 


143 


161 


108 


32 420 


51.1 


(45.3 to 56.9) 


171 


133 


176 


32 352 


52.1 


(46.7 to 57.4) 


Diabetes+obesity 


232 


1701 


65 


30 834 


19.5 


(15.9 to 23.2) 


351 


1582 


120 


30 779 


27.5 


(24.2 to 30.9) 


Stroke+smoking 


13 


142 


28 


32 649 


13.1 


(0.1 to 26.1) 


23 


132 


57 


32 620 


19.3 


(7.8 to 30.8) 


Stroke+obesity 


6 


558 


9 


32 259 


2.0 


(0.0 to 10.0) 


13 


551 


21 


32 247 


4.2 


(0.0 to 11.9) 


Smoking+obesity 


27 


447 


29 


32 329 


9.9 


(1.9 to 17.9) 


38 


436 


47 


32 311 


13.2 


(5.5 to 20.9) 



*ICD-10-AM codes: hypertension (110-115, R03.0), heart disease (I20-I52), diabetes (E10-E14), stroke (I60-I69, G45, G46), smoking (F17.2, Z72.0), obesity (E66). 
APDC, Admitted Patient Data Collection; ICD-10-AM, International Statistical Classification of Diseases and Health Related Problems 10th Revision, Australian modification. 
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but there have been limited studies of the accuracy of 
the recording of health conditions commonly used for 
case-mix or risk adjustment. Our findings regarding 
agreement for the recording of diabetes (k=0.83) were 
similar to previous Australian studies, 3 10 while agree- 
ment for hypertension (k=0.3) and heart disease 
(k=0.47) was considerably lower in our study These dif- 
ferences may be due to the fact that both previous 
studies used medical records as a 'gold standard', while 
we used self-report. Lower agreement rates for heart 
disease could be due to the broader range of heart 
disease types included in our study, with known lower 
levels of agreement for heart failure compared with myo- 
cardial infarction. 9 40 Higher sensitivities reported in a 
study from the state of Victoria 10 could also be attribut- 
able to the differences in public hospital funding 
models between the two states. Specifically, Victoria has 
used activity-based funding since 1993, while this 
method of funding was introduced in NSW and other 
Australian states only subsequent to our study period. 41 
Introduction of activity-based funding has been shown 
to increase recording of additional diagnoses and proce- 
dures in Europe. 42 

Some of the apparent discrepancies in the levels of 
coding between conditions can be attributed to the 
coding rules that govern whether or not a diagnosis is 
recorded in administrative hospital data. Additional diag- 
noses, recorded on administrative hospital data, are 
coded only if they affect the patient's treatments 
received, investigations required and/or resources used 
during the hospital stay. Thus, diagnoses that relate to 
an earlier episode, and which have no bearing on the 
current hospital stay, are not coded for that particular 
stay. Therefore, it is not surprising that (managed) 
hypertension, in particular, might not be recorded in 
hospital data relating to, e.g. elective surgery. On the 
other hand, we found that diabetes is well recorded, sug- 
gesting that it is considered to affect patient manage- 
ment in most hospital stays, and possibly reflecting the 
impact of changes to the Australian Coding Standards 
for diabetes such that between 2008 and 2010, diabetes 
with complications could be coded even where there was 
no established cause and effect relationship between dia- 
betes and the complication. 43 It is for these reasons that 
researchers using administrative data sets are encour- 
aged to incorporate information from previous hospitali- 
sations, to increase the likelihood of capturing 
morbidity, as demonstrated in this as well as other 
Australian studies. 44 

As well as looking at single morbidities, ours is the first 
study, to our knowledge, to explore the variations of 
recording of multiple conditions in hospital data. 
Concordance of two-way condition combinations was 
very low, with best results found for combinations of dis- 
eases involving diabetes, which had the highest single- 
condition level of agreement with self-reported data 
(k=0.83). Agreement measures for two-way combinations 
were found to be fair to good at best, with agreement on 
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Table 5 Variance and ICC for hospital-level random effects from multilevel logistic regression, all public and private hospitals 


in New South Wales, Australia (n=313) 




















Heart 












Hypertension 


Diabetes 


disease 


Stroke 


Smoking 


Obesity 






(N=15 279) 


(N=4794) 


(N=8307) 


(N=2480) 


(N=2099) 


(N=8162) 


Hospital-level variance (SE)* 














Model 0 


Patient factors 


O.oO (0.10) 


U.2.7 (0.0b) 


0.91 (0.1^) 


u.oo (0.10) 


O.OO (0.09) 


O.oo (0.14) 


Model 1 


Model 0 


r\ cc tr\ r\o\ 
O.OO (O.Oo) 


Kj.tLl (O.Ob) 


O./l (0.10) 


0.1 b (O.Ob) 


o.oo (o.oy) 


O.by (0.14) 




+hospital type 
















(public/private) 














Model 2 


Model 0 


0 77 (Ci C\Q\ 


0.^:0 (O.OO) 


O.y^i (O.l^i) 


0.0/ (0.10) 


O.oo (O.Oo) 


O.bo (0.14) 




+hospital 
















remoteness 














Model 3 


Model 0 


f\ Ad (C\ f\R\ 

0.4b (O.OO) 


\J.<L\J (O.Ob) 


r\ cc /r\ r\Q\ 
O.Ob (O.Oo) 


r\ oc tr\ r\Q\ 
O.^ib (O.Oo) 


r\ on tr\ no\ 
O.^y (O.Oo) 


O.bo (0.14) 




+hospital depth 
















of coding 














Model 4 


Model 0 


0.72 (0.09) 


0.21 (0.05) 


0.75 (0.10) 


0.34 (0.09) 


0.31 (0.08) 


0.67 (0.14) 




+hospital peer 
















group 














(ICC (%)f 




19.5 


7.6 


21.6 


10.4 


9.6 


17.1 


(MORf 




2.34 


1.64 


2.48 


1.80 


1.76 


2.19 


*Patient-level variance in a logistic regression is set at 


7i 2 /3=3.29 31 










tICC and MOR calculated from model 0 (ICC=hospital-level variance divided bv total variance (hospital-level+patient-level); MOR is 


calculated as 


e 0.954xVvariancej 30 














ICC, intraclass correlation coefficient; MOR, median OR; N, number of patients who self-reported condition. 







three-way condition combinations (not investigated 
here) expected to be even lower. These findings have 
implications for research into multimorbidity (the 
co-occurrence of multiple chronic or acute diseases and 
medical conditions within one person 45 ). We suggest 
that researchers who use administrative data for research 
into multimorbidity should use linked data to increase 
ascertainment and, if possible, supplement this informa- 
tion from other data sources, such as physician claims 
data or self-reported data. 

We identified considerable between-hospital variability 
in the levels of recording of common health conditions, 
with between 8% and 22% of the variation attributable 
to hospital-level factors, after adjustment for patient 
factors. This was similar in magnitude to the variability 
previously reported for performance measures (varying 
from patient satisfaction, mortality, length of stay to 



quality of care) clustered at the facility level (0-51%) 
and hospital-level variations in the use of services. 47-49 
Significant between-hospital variation was still present 
after constraining the analyses to the 82 largest public 
hospitals in the state. 

The recording of hypertension and heart disease was 
particularly variable between hospitals, those with better 
reporting having on average 2.3 and 2.5 times, respect- 
ively, the odds of recording these conditions than those 
with lower levels of reporting. The corresponding figures 
were 1.9 and 1.6 times for the 82 largest hospitals in the 
state. These findings indicate the potential for reporting 
bias to influence comparisons of health performance 
indicators between hospitals, especially for indicators that 
use conditions such as heart disease or hypertension for 
case-mix adjustment. To our knowledge, no previous 
studies have provided detailed information about how 



Figure 1 Variance for 
hospital-level random effects from 
multilevel logistic regression, for 
index and lookback admissions, 
by hospital size. *Significantly 
different from 0 at 5% level; 
**significantly different from 
0 at 1% level. 
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the validity of morbidity reporting varies among hospitals 
after accounting for patient factors. 

Furthermore, we have shown that variations in the 
accuracy of morbidity reporting between hospitals are 
predominantly driven by the hospital's depth of coding 
— concordance between self-reported and hospital data 
is lower in hospitals with a lower average number of add- 
itional diagnoses recorded. Up to 42% of the variation 
in recording at the hospital level could be attributed to 
differences in hospital depth of coding. Even though 
the measure of depth of coding we used was crude, and 
related to hospital size, it still helps in highlighting the 
impact of coding practices on variations among hospi- 
tals. Other research using the same depth of coding 
measure has shown that the lower depth of coding can 
disproportionately disadvantage hospitals' standardised 
mortality ratios, one of the commonly reported mea- 
sures of hospital performance. 2 It will be important to 
track changes in the levels of the depth of coding across 
Australian states, and to consider the implications of 
these for state-based performance comparisons, follow- 
ing the national rollout of activity-based funding and 
comparative performance reporting. 

Several factors might explain variation in depth of 
coding between hospitals. Clinical coders can code only 
information that has been recorded in the patient's 
medical record, so varying level of details recorded by 
clinicians will influence what gets coded. The training 
and professional development opportunities for coding 
staff might also influence the depth of coding. Also, 
case-mix funding systems, such as the Diagnosis Related 
Group (DRG) classification, are prone to 'upcoding' in 
order for services to receive higher reimbursement 

14 

costs. 

We found that the reporting of conditions varied with 
hospital size, larger metropolitan hospitals having higher 
concordance, with k values higher by 7% on average 
when comparing large tertiary with smaller urban hospi- 
tals. This finding echoes those of Powell et at in NSW, 
Australia, during 1996-1998 and Rangachari 16 in the 
USA, during 2000-2004. Our study showed that large ter- 
tiary hospitals had better concordance for the recording 
of hypertension and heart disease than smaller urban 
hospitals, but the reverse was true for stroke and 
smoking. Our finding that between-hospital variation in 
the recording of morbidities was up to two times higher 
when all hospitals, rather than just the largest ones, were 
included has implications for further research using data 
from smaller hospitals. This high variability in concord- 
ance among smaller hospitals may mean that 
morbidity-adjusted comparisons are not as valid as for 
larger hospitals. Researchers using information from 
these hospitals are encouraged to supplement their data 
with either self-report information and/or data linkage. 
The value-add of incorporating previous hospitalisations 
was also highlighted in our results for stroke and obesity, 
with 43-47% more patients identified using lookback 
admissions than from a single admission only. 



A particular strength of our study lies in the use of 
linked data from a large-scale cohort study to compre- 
hensively evaluate the recording of common conditions 
in hospital data, and explore the variation in recording 
among hospitals. The 45 and Up Study contains records 
for one in every 10 persons aged 45 and over in NSW, so 
it provides a rich resource to answer research questions. 
Additionally, we used advanced multilevel modelling 
methods to quantify the amount of between-hospital 
variation in the level of recording of common health 
conditions, a finding which is of importance for research 
and policy paradigms due to its impact on adjusted com- 
parisons among hospitals and the highlighted need to 
improve consistency of recording in hospitals across the 
state. To date, hospital-level variation has only been 
explored with a set outcome (e.g. mortality, readmis- 
sion) in mind. 

A potential limitation of our study was its use of self- 
reported information to explore concordance, in the 
absence of another 'gold standard', such as medical 
records. Access to medical records was not possible given 
the de-identified nature of our data and the large number 
of records in the data set. Moreover, studies that have exam- 
ined accuracy of self-reported conditions against medical 
records have found high levels of agreement, ranging from 
81 % 50 to 87% 51 for hypertension, 66% 40 to 96% 50 51 for 
diabetes and 60% 50 to 98% 52 for acute myocardial infarc- 
tion. Validation studies in the 45 and Up Study cohort have 
reported strong correlations and excellent levels of agree- 
ment between self-reported and measured height and 
weight, and derived BMI 53 as well as self-reported dia- 
betes. 54 Although the 45 and Up Study had a response rate 
of 18%, the study sample is very large and has excellent 
heterogeneity. Furthermore, exposure-outcome relation- 
ships estimated from the 45 and Up Study data have been 
shown to be consistent with a large 'representative' popula- 
tion survey of the same population. 55 

CONCLUSION 

The recording of common comorbid conditions in 
routine hospital data is highly variable and, for some 
conditions, very poor. Recording varies considerably 
among hospitals, presenting the potential to introduce 
bias into risk-adjusted comparisons of hospital perform- 
ance, especially for indicators that use heart disease or 
hypertension for risk adjustment. Furthermore, between- 
hospital variation is amplified when smaller and private 
hospitals are included in the analyses. Stratification of 
analyses according to factors that predict the complete- 
ness of recording, including hospital depth of coding 
and hospital type and size, supplementing morbidity 
information with linked data from previous hospitalisa- 
tions and increases in efforts to standardise recording 
across hospitals, all offer potential for increasing the val- 
idity of risk-adjusted comparisons. 
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